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DETAILED ACTION 

Response to Arguments 

1 . Applicant's arguments with respect to claims 1 -1 7 have been considered but are 
moot in view of the new ground(s) of rejection. 

Claim Rejections - 35 USC § 103 

2. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

3. Claims 1-4 and 6-13 and 15-17 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Wilinski et al. (WIPO Publication No. 02/095680; "Wilinski" 
hereinafter) in view of Zheng et al. (Q. Zheng, R. Chellappa; Estimation of llluminant 
Direction, Albedo, and Shape from Shading; IEEE Transactions of Pattern Analysis and 
Machine Intelligence, Vol. 13, July 1991; "Zheng" hereinafter) and Wu etal. (Z. Wu, L. 
Li; A Line-Integration Based Method for Depth Recovery from Surface Normals; IEEE, 
November 1988; "Wu", hereinafter). 

Regarding claims 1, 15, 16 and 17, Wilinski teaches a method for generating 
from a single view input image, a depth map comprising depth values representing 
distances to a viewer, for respective pixels of the image (see p. 1 , lines 1 0-1 8 for 
generating a depth map, see p. 2, lines 30-32 for a single image; see p. 8 for 
determining depth of pixels in the image by known methods) and assigning a depth 
value in a first group of depth values corresponding to the to a pixel so that pixels 
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belonging to the same image segment are assigned the same group of pixel values 
(see p. 6, lines 1 and 2 for seed values composing a pixel; see p. 8, lines 31-34 and p. 
9, lines 1 and 2 for storing the depth values of the seed pixels; see p. 10, lines 26-28 for 
assigning to a pixel the depth value of the segment it belongs to). See p. 10, lines 21- 
30 for a system and units for performing the method, including an input of the digital 
image, and for preferably using a processor provided with a computer program to 
execute the method. 

Wilinski does not explicitly teach computing cost values that comprise respective 
measures of a number of and extent of transitions in luminance and/or color 
components for pixels of the image on a path related to a spatial disposition of objects 
in the image, wherein said computing includes computing a cost value for a first one of 
the pixels of the image by determining transitions between image segments based on 
differences between luminance and/or color values of pairs of neighboring connected 
pixels at transitions which are disposed on a path from the first one of the pixels to a 
second one of the pixels wherein the second one of the pixels belongs to a 
predetermined subset of the pixels of the image. Wilinski does teach detecting 
transitions between objects in the image by contours (see p. 7, lines 29 and 30 for 
segmentation based on luminosity; see p. 5, lines 12-18 for using a method for 
detecting edges by intensity changes); and as mentioned before assigning depth to 
pixels in those edges, and through them to pixels in their segment. Zheng however 
teaches a method to obtain shape from shading using contours (see abstract; see p. 
684, 1 st col., 2 nd par.), where the value of the intensity difference across an edge is 
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determined and if it is above a threshold, detects a relative depth variation between the 
segments (see p. 684, 2 nd col., 3 rd and 4 th pars.). Because Wilinski teaches performing 
segmentation of an image and depth determination of the pixels at the contours 
generated, and Zheng teaches determining the depth of edges in the image by 
examining intensity differences across edges, it would have been obvious to one of 
ordinary skill in the art at the time of the invention to combine the image segmentation 
and depth determination taught in Wilinski with the depth determination across edges 
taught by Zheng. This combination is implicitly suggested by Wilinski, which discloses 
segmentation and depth determination of edge pixels by known methods, and expressly 
by Zheng, which discloses that the method can be improved by performing 
segmentation first; see the last three lines of page, 684, 2nd col., 3rd par. 

Neither Wilinski nor Zheng teach that the measurements for depth are taken 
along a path which results in a cost function for each pixel which is being examined for 
depth. The examiner, however, considers obvious that, since the depth determination 
disclosed in Zheng is a relative depth between the two segments delimitated by the 
contour, the only way to determine the depth for a particular pixel belonging to such a 
segment is to measure that relative depth with respect to all such segments. In order to 
do this, the examiner considers that it would have been obvious to one of ordinary skill 
in the art at the time of the invention to examine the different transitions along a path 
from the pixel to one of the edges of the image and to sum this differences which 
translate in depth differences as taught by Zheng. Reconstructing shape information 
from shading through the use of paths is well known in the art, as shown, for example in 
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Wu (see abstract). Therefore, it would have been obvious to one of ordinary skill in the 
art at the time of the invention to accumulate the differences measured as taught by 
Wilinski and Zheng along a path as taught by Wu, in order to obtain a global, rather than 
relative depth for a pixel. 

Regarding claim 2, Wilinski, Zheng and Wu further teach implicitly that the 
predetermined subset comprises one selected from the group consisting of (i) pixels 
which are located at a border of the image, (ii) pixels of a part of the border, and (iii) a 
central pixel of the image". Wu however teaches setting the reference pixel at the center 
of the image, see p. 593, Implementation subsection, 3 rd par.; where the reference point 
is used as the target to create paths and is given and assumed depth, see p. 592, par. 
below formula 2.6. The examiner considers obvious that when the image is assumed to 
have the object in the foreground as the central object in the image, which is a 
reasonable assumption for most images, the pixels at the edges of the image will more 
likely be the background and the pixel at the center will more likely be the foreground. 
Under that assumption, creating a path from the pixel being measured to the edge of 
the image would give a good approximation of height with respect to the background, 
and creating the path to the center of the image would give a good approximation to the 
foreground. Therefore, it would have been obvious to one of ordinary skill in the art to 
use or combine such paths in order to obtain a global depth value for the pixel, when no 
previous knowledge of depth is available. 
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Regarding claim 3, Wilinski, Zheng and Wu also disclose that "a first one of the 
differences is equal to a difference between respective values of neighboring pixels 
which are disposed on the path (in Zheng, as explained for claim 1 , above). 

Regarding claim 4, Wilinski, Zheng and Wu do not explicitly teach that "a second 
one of the differences is equal to an absolute value of difference between respective 
values of neighboring pixels which are disposed on the path". Wilinski teaches that the 
difference is compared to a threshold; see p. 684, 2 nd col., 3 rd par., and a depth value is 
only computed if the difference is above that threshold. Obviously, when applying a 
threshold to the difference in intensity value, the absolute value would be used, since 
the difference value could be either negative or positive. 

Regarding claim 6, Wilinski, Zheng and Wu do not explicitly teach that "the cost 
value for the first one of the pixels is computed by accumulating the differences 
between the values of the pixels which are disposed on the path". The examiner 
however considers this to be obvious, as discussed for claim 1, above. See in Wu the 
abstract for integrating the relative depths along a path. 

Regarding claim 7, Wilinski, Zheng and Wu do not explicitly teach that "the cost 
value for the first one of the pixels is computed by accumulating the differences 
between the values of the pixels which are disposed on the path", although the 
examiner considers this limitation obvious as discussed. Zheng further teaches that the 
pixel values are adjusted to meet a predetermined threshold, see discussion for claim 4 
above. 
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Regarding claim 8, Wilinski, Zheng and Wu do not teach that "the cost value for 
the first one of pixels is computed by accumulating products of differences between the 
values of the pixels which are disposed on the path and respective weighting factors for 
the differences". The examiner however considers that, since Wilinski teaches that the 
pixels belonging to a segment receive the values for that segment, as discussed above, 
and the image is segmented in order to distinguish contours delimiting surfaces of the 
objects, and as taught by Zheng, determining the values of the edge pixels for the edge 
pixels requires local information, which could diffuse the value of depth for the transition 
at the edge where the pixel belongs (see p. 684, 2 nd col., 4 th par.) one of ordinary skill in 
the art at the time of the invention would have found it obvious to accentuate the 
differences between adjoining uniform surfaces by weighting the differences so that the 
objects silhouettes would have been more clearly defined, for example by weighting 
more heavily those differences that are closer to the pixel measured, thereby increasing 
the contrast of the resulting depth image and compensating for the possible sampling 
error incurred in the sampling of neighboring pixels for edge pixels. 

Regarding claim 9, Wilinski, Zheng and Wu would have rendered obvious to one 
of ordinary skill in the art at the time of the invention that a first one of the weighting 
factors which is related to a difference between a value of a particular pixel and a value 
of its neighboring pixel, is based on a distance between the particular pixel and the first 
one of the pixels (see discussion for claim 8, above). 

Regarding claim 10, Wilinski, Zheng and Wu would have rendered obvious to 
one of ordinary skill in the art at the time of the invention that a second one of the 
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weighting factors which is related to a difference between a value of a particular pixel 
and a value of its neighboring pixel, is based on the location of the neighboring pixel 
related to the particular pixel (see discussion for claim 8, above). 

Regarding claim 1 1 , Wilinski, Zheng and Wu implicitly teach computing a second 
cost value for the first one of the pixels of the image by combining differences between 
values of pixels which are disposed on a second path from the first one of the pixels to a 
third one of the pixels which belongs to the predetermined subset of the pixels of the 
image (see Wu, p. 592, 1 st col., par. below formula 2.7 for using multiple paths for 
reducing errors in measurement). 

Although Wu teaches using the average of the relative depths obtained along the 
path to resolve conflicting cost values for the same pixel, the examiner considers 
obvious that where two or more conflicting measurements for a unique value are found, 
there are a limited number of choices on how to solve said conflict. Therefore it would 
have been obvious for one of ordinary skill in the art to modify the method disclosed in 
Wu, with a choice of the minimum value when two or more conflicting values are found 
for the same measurement to resolve this conflict by either choosing the larger value, 
the lower value, the average value or some weighted combination of both as also taught 
by Wu in the cited section. Arriving at the conclusion that the minimum value is more 
likely to be correct would have come naturally to one of ordinary skill in the art at the 
time of the invention after normal testing of the method, if, for example, it was found 
that the most frequent error in measurement is to overestimate the value for a point. 
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Regarding claim 12, Wilinski, Zheng and Wu also teach "computing a second 
cost value for a third one of the pixels on basis of the cost value for the first one of the 
pixels" (in Wilinski, see p. 1 0, lines 26-28 for assigning each pixel the pixel value of the 
segment it belongs to and discussion above for calculating the depth of the pixels in the 
contour in order to get that depth). 

Regarding claim 13, Wilinski, Zheng and Wu also teach "computing the second 
cost value by combining the cost value of the first one of the pixels with a difference 
between further values of further pixels which are disposed on a second path from the 
third one of the pixels to the first one of the pixels" (see discussion for claim 1 1 , above 
for how Wu teaches averaging the results obtained from two paths). 
4. Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over Wilinski et 
al. (WIPO Publication No. 02/095680; "Wilinski" hereinafter) in view of Zheng et al. (Q. 
Zheng, R. Chellappa; Estimation of llluminant Direction, Albedo, and Shape from 
Shading; IEEE Transactions of Pattern Analysis and Machine Intelligence, Vol. 13, July 
1991; "Zheng" hereinafter) and Wu etal. (Z. Wu, L. Li; A Line-Integration Based Method 
for Depth Recovery from Surface Normals; IEEE, November 1988; "Wu", hereinafter) as 
applied to claim 1 above, and further in view of Cahill et al. (U.S. Patent Publication No. 
2004/0062439, "Cahill" hereinafter). 

Regarding claim 5, Wilinski, Zheng and Wu also teach that "the values of pixels 
correspond to one of luminance (....)" (see discussion for claim 1, above). Wilinski, 
Zheng and Wu do not teach that the values of the pixels are measured in terms of color. 



Application/Control Number: 10/597,976 Page 10 

Art Unit: 2628 

Cahill however also teaches a method of generating a depth map where color is used in 
conjunction with luminance for segmentation (see par. [0002]). 

Because Wilinski, Zheng, Wu and Cahill teach methods of creating depth maps 
from a 2-D image, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the pixel evaluation of luminance in Wilinski, Zheng and 
Wu with the alternative evaluation of chrominance taught in Cahill because both 
techniques are interchangeable and combinable to give a more precise result, one 
applying to grey-level pictures, and the other to color pictures, and both are well-known 
in the art as taught by Cahill. 

5. Claim 14 is rejected under 35 U.S.C. 103(a) as being unpatentable over Wilinski 
et al. (WIPO Publication No. 02/095680; "Wilinski" hereinafter) in view of Zheng et al. 
(Q. Zheng, R. Chellappa; Estimation of llluminant Direction, Albedo, and Shape from 
Shading; IEEE Transactions of Pattern Analysis and Machine Intelligence, Vol. 13, July 
1991; "Zheng" hereinafter) and Wu et al. (Z. Wu, L. Li; A Line-Integration Based Method 
for Depth Recovery from Surface Normals; IEEE, November 1988; "Wu", hereinafter) as 
applied to claim 12 above, and further in view of Nakatsuna et al. (U.S. Patent 
Publication No. 2002/0154116). 

Regarding claim 14, Wilinski, Zheng and Wu further teach that "cost values 
corresponding to respective pixels of the image are successively computed on basis of 
further cost values being computed for further pixels" (see discussion for claim 1 , 
above). Wilinski, Zheng and Wu do not teach that a first scan direction of successive 
computations of cost values for a first row of pixels of the image is opposite to a second 
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scan direction of successive computations of cost values for a second row of pixels of 
the image, although Wu teaches performing the computations on a row-by-row basis 
(see p. 593, subsection Implementation, first par.). Nakatsuna, however, teaches a 
method of interpolating depth values on a pixel-by-pixel basis (see par. [01 19] and 
[0165]), in which the pixels are evaluated in a zigzag path, so that pixels may be 
positioned in a two-dimensional neighborhood (see par. [0179]). 

Because Wilinski, Zheng, Wu and Nakatsuna disclose evaluating depth for pixels 
on a row-by-row basis, it would have been obvious to one of ordinary skill in the art at 
the time of the invention to combine the depth measurement method as disclosed in 
Wilinski, Zheng, Wu, with the zigzag inspection path disclosed in Nakatsuna. Such an 
approach would be representative of the well known principle of locality in program 
optimization, by which it is advantageous to perform tasks in such an order that the 
values that have just been calculated and are therefore readily available, are those 
needed to perform the next calculation. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Carlos Perromat whose telephone number is (571 ) 270- 
7174. The examiner can normally be reached on M-TH 8:30 am- 5:00 pm EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kee M. Tung can be reached on (571)272-7794. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 



Application/Control Number: 10/597,976 Page 12 

Art Unit: 2628 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Kee M Tung/ /Carlos Perromat/ 

Supervisory Patent Examiner, Art Unit 2628 Examiner 
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